Incremental learning based on non-incremental in- duction algorithm

The machine learning algorithms can be divided into two general types: non-incremental that processes all training examples at once and incremental that handles examples one by one. This paper describes the multi-layer incremental inference algorithm (MLII) [1] based on the non-incremental inductive inference algorithm CN2 [2]. In original, the MLII algorithm used linked with the non-incremental algorithm HCV [3], which is based on the extension matrix approach, works with symbolic variables and as a result generates conjunctive formula [3]. The MLII algorithm consists of three steps [1]: (1) data partitioning (divide initial data in subsets of approximately same size, shuffling data randomly); (2) generalization of rules learned from the first subset; and (3) reduction of a set of previously obtained rules in order to produce more accurate and consistent rules. The multi-layer incremental induction divides an initial training set into subsets of approximately equal size, runs an existing induction algorithm on the first subset to obtain a first set of rules, and then processes each of remaining data subsets [1] at a time by including the induction results from the previous subsets. In such a way, the multi-layer induction accumulates the found rules from each data subset at each layer and produces final integrated output. As layers, the data subsets are described-using layers the effects of noise would be diluted and induction efficiency can be increased [1]. The paper provides practical experiments with the MLII algorithm that consider e-mail messages classification task. Basic e-mail dataset is processed by Levenshtein Distance [4] to receive most frequently used words in e-mail Subject and Body. The learning set examples are randomly shuffled and divided into subsets using data partitioning method. From the data subsets rules are generated using free software Sipina for Windows-Research version [5] exploiting inductive inference algorithm CN2 with the following parameters: rule evaluation function-Laplacian error rate, significance level-0.001. Experiments are performed to study working principles of the MLII algorithm , to compare algorithm CN2 usage in MLII and interface agent in e-mail messages filtering task-practical experiments of what is described in